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ABSTRACT 

Background: Systematic reviews are recognized as the most effective means of summarizing research evidence. 
However, they are limited by the time and effort required to keep them up to date. Wikis present a unique opportunity 
to facilitate collaboration among many authors. The purpose of this study was to examine the use of a wiki as an online 
collaborative tool for the updating of a type of systematic review known as a scoping review. 

Methods: An existing peer-reviewed scoping review on asynchronous telehealth was previously published on an 
open, publicly available wiki. Log file analysis, user questionnaires and content analysis were used to collect descrip- 
tive and evaluative data on the use of the site from 9 June 2009 to 10 April 2010. Blog postings from referring sites 
were also analyzed. 

Results: During the 10-month study period, there were a total of 1222 visits to the site, 3996 page views and 875 
unique visitors from around the globe. Five unique visitors (0.6% of the total number of visitors) submitted a total of 
6 contributions to the site: 3 contributions were made to the article itself, and 3 to the discussion pages. None of the 
contributions enhanced the evidence base of the scoping review. The commentary about the project in the blogosphere 
was positive, tempered with some skepticism. 

Interpretations: Despite the fact that wikis provide an easy-to-use, free and powerful means to edit information, 
fewer than 1% of visitors contributed content to the wiki. These results may be a function of limited interest in the 
topic area, the review methodology itself, lack of familiarity with the wiki, and the incentive structure of academic 
publishing. Controversial and timely topics in addition to incentives and organizational support for Web 2.0 impact 
metrics might motivate greater participation in online collaborative efforts to keep scientific knowledge up to date. 
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SYSTEMATIC REVIEWS HAVE BECOME THE MOST 
effective means of identifying, selecting, assessing 
and synthesizing all original research evidence 
relevant to a specific question. Most systematic reviews 
have finite life spans, with a median survival (time from 
publication until availability of new information that 
potentially changes the effect size or direction) of 5.5 
years; it has been estimated that, for 23% of systematic 
reviews, new information is available within 2 years of 
publication. 1 Despite the need to keep reviews current, 
only 17.7% of systematic reviews seem to be cited as up- 
dated versions of previously published reviews. 1 ' 2 

Given its open, transparent nature and the availabil- 
ity of free software for online collaboration, the Internet 
presents a unique opportunity to explore new ways to 
update and publish academic research. Wikis, in par- 
ticular, are efficient tools for co-creating, maintaining 
and making widely available repositories of knowledge. 
In general, a wiki allows anyone to easily add to, edit 
or delete the content of a website. The wiki application 
keeps track of all changes and offers the ability to ac- 
knowledge author contributions. It has been suggested 
that some of the benefits of using a wiki for collabora- 
tive authoring include the ability to track authorship, 
monitor the development of an article, reduce conflicts 
of interest and update it swiftly, thus providing a current 
evidence resource. 3-5 

Wikis are used very successfully as highly efficient 
tools to co-create and maintain large amounts of gen- 
eral-interest content. A case in point is Wikipedia, the 
largest encyclopedia in the world. With over 3.5 mil- 
lion pages of articles in English alone, each of which is 
edited 19 times on average, and over 13 million regis- 
tered users, 6 this free resource overtook all other en- 
cyclopedias in size 7 and possibly even the Encyclopedia 
Britannica in quality, in less than 5 years. 8 Wikipedia, 
which contains over 20 000 health-related articles, has 
become a popular health resource for the general pub- 
lic and the medical community. 5 ' 9 The top 200 medical 
articles on Wikipedia receive, on average, over 100 000 
page views per month, 10 and studies suggest that 50% to 
70% of practising physicians use Wikipedia as a resource 
when providing care. 5 ' 11 ' 12 

Although Wikipedia is by far the largest and most 
popular collaboratively authored reference website, 
there are now over 70 health-related wikis in existence 
of varying specialties and depth. 13 Some serve as a gen- 
eral medical reference (e.g. Ganfyd.org) or subspecialty 
resource (e.g. Wikisurgery.org) for health professionals, 
while others are intended as a shared knowledge base for 
people with specific health concerns or conditions (e.g. 



Wikicancer.org). Research is underway to explore the 
utility of wikis as a knowledge base and a pedagogical 
tool. 14 ' 15 To the best of our knowledge, however, there 
have been no published research efforts to explore the 
role that wikis could play in relation to updating and 
maintaining peer-reviewed systematic reviews of health 
interventions. This study was designed to begin to ad- 
dress this gap. 

The purpose of this study was to examine the use 
of a wiki as an online collaborative tool for updating a 
peer-reviewed scoping review. A scoping review is a 
type of systematic review that provides a comprehen- 
sive summary of the existing literature or evidence base 
on a broad topic. 16 ' 17 Similarly to traditional systematic 
reviews, scoping reviews follow rigorous and transpar- 
ent methods to identify all relevant literature. 16 Unlike 
a full systematic review, however, they typically do not 
provide a quality assessment of included studies or ex- 
tensive data synthesis. 17 A peer-reviewed scoping review 
was adapted for publication on an open, public wiki. This 
article describes the characteristics of the users, their 
use of the site, and the nature, quality and quantity of 
the contributions, as well as the "buzz" that accumulated 
on the Internet during the study period. 

Methods 

The open, public wiki was created and made available 
through the Open Medicine web site from 2 June 2009 
to 10 April 2010, under the name Open Medicine wiki 
Chttp://wikisr.openmedicine.ca/index.php/Main_Page). 

The wiki was populated with a pre-existing, peer- 
reviewed scoping review on asynchronous telehealth by 
Deshpande and colleagues 18 that was initially published 
by CADTH in print and online form in January 2008, 
after peer review. 19 In preparation for its publication for- 
mat on the Open Medicine wiki, the review underwent 
a second round of peer review by the editorial board of 
Open Medicine. 

The Open Medicine wiki utilized the open-source 
platform developed by Mediawiki (the same software 
application used by Wikipedia) to create a viewing and 
navigating experience that would likely be familiar to 
users of the site (Fig. 1). The wiki was hosted and admin- 
istered by the Centre for Global eHealth Innovation, a 
joint research initiative of the University of Toronto and 
the University Health Network in Toronto. It was brand- 
ed with the Open Medicine logo and linked to the Open 
Medicine website. 

Initially, the Open Medicine wiki was launched as a 
public, "unrestricted" wiki open for anyone to view, add, 
change or delete content, and join. Registration was not 
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required to contribute or modify content on the site, in 
an effort to lower any perceived barriers to participation. 
In response to extensive spamming, however, 2 spam 
filters (ReCAPTCHA and SpamBlacklist) were installed 
on the site on 24 June 2009, which served to effectively 
prevent further inappropriate contributions to the site. 

As of 21 July 2009, anyone interested in contributing 
or modifying content on the site was required to regis- 
ter with a unique log-in ID and password, answer a few 
basic demographic questions (e.g., age, gender, country 
of residence, experience with wikis), and a competing 
interest statement. Registered users also had access to 
the discussion page of each site page, which served as an 
open forum for dialogue and interaction with other users 
of the site. There was no pre-screening of edits or contri- 
butions to the wiki. The only changes to the content of 
the site made by the site administrator (LS) were "roll- 
backs" to previous versions when the site was spammed 
with inappropriate and unrelated content. 

The Open Medicine wiki was promoted by the jour- 
nal Open Medicine in 2 eBulletins (2 June 2009 and 
21 July 2009) and a Rapid Response article (14 August 
2009) that were published on the Open Medicine site and 
distributed to members of their email list, as well as by 
the Canadian Society of Telehealth (now known as the 
Canadian Telehealth Forum) in an eBulletin (30 October 
2009) published on their site and distributed to members 
of their email list and through personal communication 
with their Research Committee (1 December 2009). 

Data collection 

A mixture of quantitative and qualitative methods 20 was 
used to collect descriptive and evaluative data on the use 
of the site from 9 June 2009 to 10 April 2010. 

Google Analytics, a service created by Google to meas- 
ure various metrics related to website usage, was used to 
summarize log file data on how the wiki was used. The 
following parameters were evaluated: 

• number of site visits (based on unique IP address) 

• number of new visits (first visit from unique IP 
address) 

• number of unique site visitors (visits from unique 
IP address) 

• country of domicile of site visitors (where Internet 
Service Provider is registered) 

• number of total and average page views (specific 
web site page) 

• source of traffic (referring web site) 

• direct traffic (visitors who typed in the URL of the 
site in their browser, or accessed the site from a 
browser bookmark, link in email or documents) 
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• bounce rate (frequency of times a visitor lands on 
the first page and then exits the site) 

These usage statistics were collected from any visitor to 
the site regardless of whether he or she registered to edit 
the wiki. 

Visitors who chose to edit the wiki were required 
to complete a brief online questionnaire as part of the 
registration process. This questionnaire collected infor- 
mation on: 

• gender and age 

• country of residence (where the visitor was living 
at the time of site access) 

• experience using wikis (whether the visitor had 
ever obtained information from or contributed 
content to a wiki) 

• referral source (e.g., journal, professional society, 
browser, accidentally, word of mouth) 

• institutional affiliation 

• role (e.g., health care worker, researcher, adminis- 
trator, patient, member of the public, etc.) 

• competing interests (e.g., employment, personal or 
financial) 

Content analysis was used to evaluate and describe the 
nature and quantity of contributions or modifications to 
the content of the site, including messages posted to the 
site discussion pages, as well as the relevant talk in the 
"blogosphere," which was limited to sites from which in- 
dividuals were referred to the Open Medicine wiki. 

Results 

Site traffic. During the 10-month study period, there 
were a total of 1222 visits to the site, 3996 page views 
and 875 unique visitors (Table 1). Visitors came from 66 
different countries, with 72.2% of visits originating in 
Canada or the United States (Table 2). The majority of 
site visits occurred during the first 2 months after the 
wiki launch. Similar proportions of visits came from 
referring sites (40.4%) and from direct traffic (39.4%) 
(Fig. 2). 

In total, there were 55 unique referring sites (e.g., 
websites that included a link to the Open Medicine wiki). 
The Open Medicine website was responsible for nearly 
50% of the referred traffic: 167 referrals originated from 
a link within the article on the Open Medicine website 
and 63 from the Open Medicine blog. Of the top 10 refer- 
ring websites, 6 were blogs (Table 3). 

Site users. In total, there were 61 user accounts; however, 
only 13 were created by "genuine" users. The remaining 
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21 accounts were deemed to be "fake" accounts by the 
team's site administrator (LS) created by either auto- 
mated scripts or individuals entering malicious informa- 
tion that was either false or nonsensical. An example of a 
"fake" or fabricated account is shown in Figure 3. 

Of the 13 genuine accounts, 5 were created by mem- 
bers of the project team and 8 were created by non-team 
members (one individual created two separate accounts). 
Only 4 of the 7 unique non-team members who regis- 
tered with the site completed the registration question- 
naire (three registered before the addition of the survey). 
Of these 4, 3 were male, 3 were Canadian, all were be- 
tween 30 to 60 years old, all had previously used a wiki 
for information, 3 were affiliated with a university and 
2 were health researchers, 1 was a health administrator 
and 1 was a health practitioner. Only 1 had previously 
contributed content to a wiki, and only 1 reported having 
a conflict of interest. 

Site contributions. Of the 875 people who visited the 
site, 5 made a total of 6 contributions to the site (0.57% 
of the total number of unique visitors to the site). 

The 3 contributions to the article itself were as follows: 

• A sentence about the function of asynchronous tele- 
health was added to the abstract; 

• Competing interests were added to the author dec- 
laration section; and 
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• A grammatical change was made to the content in 
the competing interests section. 

Two posts were added to the article's discussion page. 
One contributor posed the question about the exist- 
ence of safeguards to protect patient privacy, which was 
answered by the project team. The second contributor 
suggested expanding the definition of telehealth used 
in the article to include online support communities for 
patients and health care professionals. Neither of these 
posts generated responses or commentary from other 
users of the site. Lastly, one post requesting help with 
the registration process was added to the site's help page. 

Buzz in the blogosphere 

This project generated some comments on blog sites. In 
general, the blog commentary about the project was posi- 
tive, but it was tempered with some skepticism. The posi- 
tive blog postings applauded the initiative and the door 
the wiki trial opened to further experimentation and in- 
novation in the field of academic publishing. Some of the 
most representative positive comments were as follows: 

The technophile blurry-eyed visionary in me is very 
impressed. Congratulations to this sister discipline 
[commenter was from law] for having the courage and 
foresight to lead the way— Blogger 1 
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... a simple idea with significant consequences. It should 
enable risk-free experimentation with all sorts of web 
2.0 innovations, social networking, and collaborative 
research and writing. Some will fail to add value. That 
doesn't matter. The point is not that all experiments 
will succeed but that this simple idea frees us to 
experiment— Blogger 2 

It will be interesting to see how successful this 
approach is and to speculate about how the potential 
changes from this publishing model will play out in the 
future— Blogger 3 

Negative blog posts revolved around concerns about 
threats to the quality of the article if non-experts were 
allowed to contribute to it and the sustainability of 
such an initiative. Two such examples are as follows: 

"The law prof, writer, editor and publisher in me is 
writhing at the fate [that] could befall an article at the 
hands of the public, even a public that has to register 
first"— Blogger 1 

"I think the wiki is worth a try, but I worry about its 
staying power. I often have trouble getting docs to use 
PubMed properly, so how can I convince them to build a 
wiki?"— Blogger 4 

Discussion 

To our knowledge, this is the first attempt to examine 
the use of an open, publicly available wiki as a tool to 
engage the academic and clinical communities, as well 
as the public, in the collaborative updating process for a 
peer-reviewed scoping review. Our findings indicate that 
this initiative failed to add value to the body of evidence 
on asynchronous telehealth. Although the wiki attracted 
875 visitors from around the globe, fewer than 1% con- 
tributed content to the article and none of the contri- 
butions substantially improved the nature or extent of 
evidence presented in the article. However, this initiative 
succeeded in stimulating dialogue about novel formats 
of academic publishing, as evidenced by the commentary 
that accumulated in the blogosphere. It also provided a 
lesson in what does not work, and in doing so has alert- 
ed us to important contextual factors that may hinder 
the role of crowd-sourcing through wikis as a means to 
keep science up to date. 

Although there is a wide spectrum of activity in on- 
line communities, most have a handful of dedicated 
contributors and a much larger number of readers, also 
commonly known as lurkers (people who read but do not 
post). The average ratio of posters-to-readers in mailing 
lists and message boards has been estimated at ioo:i. 21 
The main reasons for not actively participating in these 



online communities include not needing to post, or feel- 
ing as though one has nothing to contribute; needing 
to find out more about the group before participating; 
thinking that they were being helpful by not posting; 
not understanding how the software operates; and not 
liking the dynamics of the community. 22 In the case of 
Wikipedia, there has been debate concerning whether its 
success is due to "the power of the few or the wisdom of 
the crowd." 23 ' 24 Research suggests that a small propor- 
tion of editors account for most of the work done and val- 
ued added. 24 " 26 

The levels of contributions to the wiki in the present 
study could not be explained by the absence of new 



Table 1: Site traffic 


Type of traffic 


Count or rate 


Total site visits 


1222 


Unique site visitors 


875 


Total page views 


3996 


Average page views 


3.27 


Bounce rate 


52.0% 


New visitors 


70.3% 




Table 2: Geographic source of traffic 


Country 


Count (%) 


Canada 


636 (52.0) 


USA 


237(19.4) 


UK 


50(4.1) 


Australia 


36 (2.9) 


Brazil 


25 (2.0) 


Spain 


21 (1.7) 


Argentina 


17(1.2) 


Germany 


15 (1.2) 


Netherlands 


14(1.1) 


India 


14(1.1) 




Table 3: Top 1 0 referring sites 


Country 


Count (%) 


Canada 


636 (52.0) 


United States 


237(19.4) 


United Kingdom 


50(4.1) 


Australia 


36 (2.9) 


Brazil 


25 (2.0) 


Spain 


21 (1.7) 


Argentina 


17(1.2) 


Germany 


15(1.2) 


Netherlands 


14(1.1) 


India 


14(1.1) 
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evidence, as there were at least 6 articles just on tele- 
dermatology published between the third week of Nov- 
ember of 2006, 27-32 when the original search of the 
literature was performed, and the end of the data col- 
lection period in April 2010. It is possible that there was 
limited interest in the relatively narrow topic area of 
asynchronous telehealth; however, targeted efforts were 
made to increase awareness about its existence among 
hundreds of specialist members of the Canadian Society 
of Telehealth. It is more likely that the results are a func- 
tion of the review methodology itself, lack of familiarity 
with the wiki, and the incentive structure of academic 
publishing. 5 ' 33 

As discussed earlier, only 17.7% of systematic reviews 
seem to be cited as updated versions of previously pub- 
lished reviews. 1 ' 2 This low update rate could, in part, 
reflect the effort required to complete a systematic re- 
view. Updates are time-consuming, expensive and yield 
relatively little new evidence if conducted too often. 34 
Although updating a scoping review typically involves 
less effort than a traditional systematic review given the 
lower level of data integration, similar logistical barriers 
could have discouraged users of the wiki from updating 
the present review. 

Similarly, perceived ease of use— a key determinant 
of technology adoption 35 could have deterred some users 
from editing the wiki. We know that positive outcome 
expectancy, previous use of information technology and 
Internet self-efficacy influence the use of the Internet for 
health information more generally. 36 Current estimates 
indicate that only 10% of physicians have edited one or 
more articles on Wikipedia, 12 which suggests a relatively 
inexperienced user population. Recent research suggests 
that certain socialization tactics (e.g., welcome messa- 
ges, assistance and constructive criticism) can enhance 
users' confidence in participating in a wiki. 37 




Figure 2: Traffic sources 



By far the most important determinant of technology 
adoption is perceived usefulness. 35 Unfortunately, recent 
initiatives suggest a general lack of interest among schol- 
ars in Web 2.0 publishing. 33 Some journals like Open 
Medicine (e.g., BMJ, PLoS, Nature, Bioinformatics) have 
been experimenting with article commenting, which 
many have already considered to be a failure for not at- 
tracting enough participation. 33 ' 38 ' 39 

According to Priem and Hemminger, 33 the lack of par- 
ticipation of scholars in article commenting, which we 
propose could be extended to all Web 2.0 scholarly activ- 
ities, can be attributed to "the three 'C's': culture, credit, 
and critical mass." There are major cultural barriers to 
the use of Web 2.0 tools in scholarly publishing. This 
kind of work is not encouraged in medical or postgrad- 
uate education, and does not count toward publications, 
grants or any other type of currency for career advance- 
ment. However, to achieve sustainability, Web 2.0 initia- 
tives must establish a critical mass of contributors. 5 ' 33 
High levels of activity also lend credibility by implying 
that the site is popular, which in turn motivates more 
people to contribute content. 40 

Heilman and colleagues 5 have provided a number of 
suggestions that could facilitate the adoption of wikis by 
building a culture of open, collaborative authoring in sci- 
ence and medicine. These include: requiring students to 
critically appraise and improve articles on Wikipedia as 
part of their medical or postgraduate education curric- 
ula; encouraging participation in WikiProject Medicine 
(a community of Wikipedia editors interested in improv- 
ing the quality of health-related articles), which is apply- 
ing for credit as a continuing medical education (CME) 
opportunity; and the coupling of traditional publishing 
with contributions to Wikipedia as demonstrated by the 
scientific journal RNA Biology. 41 



Profile question 


Response 


Affiliation (text box) 


kSSflAjlWFymGv 


Role (menu options) 


Public 


Competing interests (Y/N) 


No 


Specify competing interests (text box) 


JELvOHbhl 


Sex (M/F) 


Male 


Age (menu options) 


15-20 


Location (menu options) 


Holy See (Vatican City State) 


Experience with reading wikis (Y/N) 


Yes 


Experience with editing wikis (Y/N) 


Yes 


How did you hear about his wiki (text box) 


aslNLbuzuLiMNbAXS 



Figure 3: Example of fake account profile 
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Ultimately, in order for wikis to be a viable model for 
collaborative publishing in academia measures must be 
in place to recognize and credit author contributions. 
Several online platforms are currently testing tools and 
strategies to recognize authorship (e.g., WikiGenes, 
MedPedia, Google Knol). 3 One relevant example is OPI- 
MEC (www.opimec.org/), an observatory of innovative 
practices in complex, chronic disease management, 
which produced the first collaboratively authored book 
on poly-pathology using online crowd-sourcing efforts. 
As a platform designed purposefully to bring together 
like-minded individuals, OPIMEC met all of Heilman's 
Cs: it already had a strong culture of collaboration, a 
critical mass of scholars knowledgable in the topic area, 
and tools and processes for the identification and recog- 
nition of contributions (e.g., lead authors produced the 
first version of each chapter, integrated contributions 
from the community and credited community mem- 
bers as co-authors on the basis of the extent of their 
contributions). 42 

There is also growing interest in new ways to measure 
scientific impact. 33 As discussed in detail by Priem and 
Hemminger, "these days scholars not only cite previous 
work in journal publications, they may add it to their 
personal web pages, bookmark, tweet, or blog about it." 33 
Building one's online research identity may soon become 
a necessity for researchers. Evaluators could use Web 
usage tracking tools to mine and evaluate these social 
media-based metrics of scholarly impact. 

Future research should examine whether these and 
other strategies to recognize distributed knowledge dis- 
semination efforts could improve the viability of wikis in 
academic publishing. Whether this is the case could be 
easily explored through the replication of the current ex- 
periment using a review on controversial interventions 
for a highly prevalent condition, with a rapidly growing 
body of research, and whose conclusions could have an 
important impact on a large number of health profession- 
al, academic, policy and corporate groups. By changing 
the study in this manner and by incorporating strategies 
to recognize contributions, it might be possible to estab- 
lish whether wikis could in fact be valuable resources for 
keeping research evidence up to date. Ultimately, we may 
need the buy-in of the academic community on a new set 
of scholarly impact metrics that take into account these 
new forms of knowledge dissemination. 



Contributors: ARJ and AD conceived the project. All authors contrib- 
uted to the design of the project. JLB and LS implemented the proj- 



ect, monitored the wiki, and collected the data. JLB analyzed the 
data and wrote the first draft of the manuscript. All authors contribut- 
ed to the manuscript and approved the final version. JLB is guarantor. 

Acknowledgements: We thank the Open Medicine team for their 
assistance during the planning and implementation of the study, 
including allowing us to brand the project site with their logo and link 
to the Open Medicine homepage. 



References 

1. Shojania KG, Sampson M, Ansari, MT, Ji J, Doucette S, Moher 
D. How quickly do systematic reviews go out of date? A survival 
analysis. Ann Intern Med 2007;l47(4):224-233. 

2. Moher D, Tetzlaff J, Tricco AG, Sampson M, Altman DG. Epi- 
demiology and reporting characteristics of systematic reviews. 
PLoSMed 2007;4(3):e78. 

3. Hoffmann R. A wiki for the life sciences where authorship mat- 
ters. Nat Genet 20o8;4o(9):i047-i05i. 

4. Schmidt J. Flexibility in wiki publishing: author desires, peer 
review and citation. Academic Publishing Wiki. 2005. Available 
from: http://academia.wikia.com/wiki/Flexibility_in_wiki_ 
publishing:_author_desires,_peer_review_and_citation (ac- 
cessed 2011 April 5). 

5. Heilman JM, Kemmann E, Bonert M, Chatterjee A, Ragar B, 
Beards GM, et al. Wikipedia: a key tool for global public health 
promotion. J Med Internet Res 20ll;l3(l):ei4. 

6. Wikipedia.org. Statistics. Wikipedia: the free encyclopedia; 2011. 
Available from: http://en.wikipedia.Org/wiki/Special:Statistics 
(accessed 2011 Apr 4). 

7. Wikipedia.org. Wikipedia: size comparisons. Wikipedia: the 
free encyclopedia; 2011. Available from: http://en.wikipedia.org/ 
wiki/Wikipedia:Size_comparisons#Comparison_of_encyclo- 
pedias (accessed 2011 Apr 4). 

8. Giles J. Internet encyclopaedias go head to head. Nature 
2005;438(707o):aoo-90i. 

9. Laurent MR, Vickers TJ. Seeking health information online: Does 
Wikipedia matter? JAMA 2009;i6(4):47i-479. 

10. Wikipedia.com. WikiProject Medicine: Popular pages. 2011. 
[cited 2011 September 7] Available from: http://en.wikipedia.org/ 
wiki/Wikipedia:WikiProject_Medicine/Popular_pages 

11. Hughes B, Joshi I, Lemonde H, Wareham J. Junior physician's 
use of Web 2.0 for information seeking and medical education: a 
qualitative study. Int J Med Inform 2009:78(103:645-655 

12. Comer B. Docs look to Wikipedia for condition info: 
Manhattan research. Medical Marketing & Media. 2009; 
Apr 21. Available from: www.mmm-online.com/docs-look- 
to-wikipedia-for-condition-info-manhattan-research/ 
article/131038/ (accessed 2011 Sept 7). 

13. Rothman D. List of medical wikis. davidrothman.net; 2009. 
Available from: http://davidrothman.net/list-of-medical-wikis/ 
(accessed 2011 Apr 4). 

14. Varga-Atkins T, Dangerfield P, Brigden D. Developing profession- 
alism through the use of wikis: A study with first-year under- 
graduate medical students. Med Teach 20lO;32(lo):824-829. 

15. Brohee S, Barriot R, Moreau Y. Biological knowledge bases using 
Wikis: combining the flexibility of Wikis with the structure of 
databases. Bioinformatics 20lo;26(l7):22lo-22ll. 

16. Arksey H, O'Malley L. Scoping studies: Toward a methodological 
framework. IntlJ Soc Res Methodol 2005;8(l):l9-32. 

17. Armstrong R, Hall BJ, Doyle J, Waters E. 'Scoping the scope' of a 
Cochrane review. J Public Health 2Oll;33(l):l47-l50. 



Open Medicine 201 1 ;5(4)e207 



RESEARCH 



18. Deshpande A, Khoja S, Lorca J, McKibbon A, Rizo C, Husereau D, 
Jadad AR. Asynchronous telehealth: A scoping review of analytic 
studies. Open Med 2009;3(2):e69-e9i. 

19. Deshpande A, Khoja S, Lorca J, McKibbon A, Rizo C, Jadad AR. 
Asynchronous telehealth: Systematic review of analytic studies 
and environmental scan of relevant initiatives. Technology Re- 
port No. 101. Canadian Agency for Drugs and Technologies in 
Health: Ottawa; 2008. 

20. Creswell JW. Research design: qualitative, quantiative and 
mixed method approaches. California: Sage Publications; 2003. 

21. Nonnecke B, Preece J. Lurker demographics: counting the silent. 
In: Proceedings of the Conference on Human Factors in Comput- 
ing Systems (CHI); 2000 Apr 1-6; The Hague, Netherlands. New 
York: ACM Press; 2000. 

22. Preece J, Nonnecke B, Andrews D. The top five reasons for lurk- 
ing: improving community experiences for everyone. Comput 
Hum Behav 2004;20(2):20l-223. 

23. Panaciera K, Halfaker A, Terveen L. Wikipedians are born, not 
made. In: Proceedings of GROUP; 2009 May 10-13; Sanibel Is- 
land, Florida. New York: ACM Press; 2009. 

24. Kittur A, Chi EH, Pendleton BA, Suh, B, Mytkowicz T. Power of 
the few vs. wisdom of the crowd: Wikipedia and the rise of the 
bourgeoisie. In: Proceedings of the Conference on Human Fac- 
tors in Computing Systems (CHI); 2007 Apr 28-May 3; San Jose, 
California. New York: ACM Press; 2007. 

25. Kittur A, Kraut RE. Harnessing the wisdom of crowds in 
Wikipedia: quality through coordination. In: Proceedings of the 
Conference on Computer Supported Cooperative Work (CSCW); 
2008 Nov 8-12; San Diego, California. New York: ACM Press; 
2008. 

26. Priedhorsky R, Chen J, Lam SK, Panaciera K, Terveen L, Ridel J. 
Creating, destroying and restoring value in Wikipedia. In: Pro- 
ceedings of GROUP; 2009 May 10-13; Sanibel Island, Florida. 
New York: ACM Press; 2009. 

27. Pak H, Triplett CA, Lindquist JH, Grambow SC, Whited JD. Store- 
and-forward teledermatology results in similar clinical outcomes 
to conventional clinic-based care. J Telemed Telecare 2007;l3(l): 
26-30. 

28. Heffner VA, Lyon VB, Brousseau DC, Holland KE, Yen K. Store- 
and-forward teledermatology versus in-person visits: a compari- 
son in pediatric teledermatology clinic. J Am Acad Dermatol 
2009;6o(6):956-96i. 

29. Edison KE, Ward DS, Dyer JA, Lane W, Chance L, Hicks LL. 
Diagnosis, diagnostic confidence, and management concordance 
in live-interactive and store-and-forward teledermatology com- 
pared to in-person examination. Telemed J E Health 2008;l4(9): 
889-895- 

30. Fabbrocini G, Balato A, Rescigno O, Mariano M, Scalvenzi M, 
Brunetti B. Telediagnosis and face-to-face diagnosis reliability 
for melanocytic and non-melanocytic 'pink' lesions. J Eur Acad 
Dermatol Venereal 2008;22(2):229-234. 

31. Ferrandiz L, Moreno-Ramirez D, Nieto-Garcia A, Carrasco R, 
Moreno-Alvarez P, Galdeano R, et al. Teledermatology-based 
presurgical management for nonmelanoma skin cancer: a pilot 
study. Dermatol Surg 2007;33(9):1092-1098. 

32. Verma M, Raman R, Mohan RE. Application of tele-ophthalmol- 
ogy in remote diagnosis and management of adnexal and orbital 
diseases. Indian J Ophthalmol 2009;57(5):38l~384. 



BENDER ET AL. 



33. Priem J, Hemminger BM. Scientometrics 2.0: toward new metrics 
of scholarly impact on the social Web. First Monday. 2Ol0;l5(7). 
Available from: www.uic.edu/htbin/cgiwrap/bin/ojs/index.php/ 
fm/article/view/2874/2570 (2011 Sept 7). 

34. Chapman A, Middleton P, Maddem G. Early updates of a system- 
atic review— a waste of resources? In: Proceedings of the 4th Sym- 
posium on Systematic Reviews: Pushing the Boundaries; 2002 Jul 
2-4; Oxford,UK. 

35. Davis FD. Perceived usefulness, perceived ease of use, and 
user acceptance of information technology. Manage Inf Sci Q 
I989;i3(3):3i9"339- 

36. Mead N, Varnam R, Rogers A, Roland M. What predicts patients' 
interest in the Internet as a health resource in primary care in 
England? J Health Serv Res Policy 2003;8(l):33-39. 

37. Choi B, Alexander K, Kraut RE, Levine JM. Socilization tactics in 
Wikipedia and their effects. In: Proceedings of the Conference on 
Computer-Supported Cooperative Work (CSCW); 2010 Feb 6-10; 
Savannah, Georgia. New York: ACM Press; 2010. 

38. Nielsen M. Doing science in the open. Physics World 2009:1:30- 
35- 

39. Neylon C, Wu S. Article-level metrics and the evolution of scientific 
impact. PLoS Biol 2009;7(ll):ei000242. 

40. Flanagin AJ, Metzger MJ. Digital media and youth: unparalleled 
opportunity and unprecedented responsibility. In: Flanagin AJ, 
Metzger MJ, editors. Digital media, youth, and credibility. Cam- 
bridge (MA): MIT Press; 2008. 

41. Butler D. Publish in Wikipedia or perish. Nature News 2008; Dec 
16. Available from: www.nature.com/news/2008/081216/full/ 
news.2008.1312.html (accessed 2011 Sept 7). 

42. Jadad AR, Cabrera A, Martos F, Smith R, Lyons RF. Why mul- 
tiple chronic diseases? Why now? What is going on around the 
world? In: Jadad AR, Cabrera A, Martos F, Smith R, Lyons RF. 
When people live with multiple chronic diseases: a collaborative 
approach to an emerging global challenge. Granada: Andalusian 
School of Public Health; 2010. Available at: www.opimec.org/equi- 
pos/when-people-live-with-multiple-chronic-diseases. 



Published: 20 December 2011 

Citation: Bender JL, O'Grady LA, Deshpande A, Cortinois AA, Saffie 
L, Husereau D, et al. Collaborative authoring: a case study of the use 
of a wiki as a tool to keep systematic reviews up to date. Open Med 
2011;5(4):e201-e208. 

Copyright: Open Medicine applies the Creative Commons Attribution 
Share Alike License, which means that anyone is able to freely copy, 
download, reprint, reuse, distribute, display or perform this work and 
that authors retain copyright of their work. Any derivative use of this 
work must be distributed only under a license identical to this one 
and must be attributed to the authors. Any of these conditions can 
be waived with permission from the copyright holder. These condi- 
tions do not negate or supersede Fair Use laws in any country. For 
more information, please see http://creativecommons.org/licenses/ 
by-sa/2.5/ca/. 



Open Medicine 201 1 ;5(4)e208 



